Using Image-Processing Settings to Determine an Optimal Operating Point for Object Detection on Imaging Devices

This publication describes techniques and processes for using image-processing settings (e.g., Auto-Exposure (AE), Auto-Focus (AF), and/or Auto-White Balance (AWB)) to determine an optimal operating point for object detection by an object detector on an imaging device. An operating point is provided to the object detector by a manufacturer to enable the object detector to execute object detection. Through object detection, the object detector determines if an object is identified in the scene based on a confidence score. The optimal operating point has a computed image-processing setting that is closest to an ideal value of the image-processing setting. In an example, a fixed penalty function allows an optimal operating point to be determined using computed AE results for the image at different operating points compared to an ideal AE for the image. The smallest difference between the computed AEs and ideal AE corresponds to the optimal operating point for the image. The process can be repeated for many images to determine an optimal operating point across many types of images. Additionally, the process can be conducted with other image-processing settings, such as AF and AWB, to guide the selection of an optimal operating point across many settings. The determined optimal operating point can be provided to an object detector on an imaging device to provide a positive user experience with the imaging device

MoSculp: Interactive Visualization of Shape and Time

We present a system that allows users to visualize complex human motion via 3D motion sculptures---a representation that conveys the 3D structure swept by a human body as it moves through space. Given an input video, our system computes the motion sculptures and provides a user interface for rendering it in different styles, including the options to insert the sculpture back into the original video, render it in a synthetic scene or physically print it. To provide this end-to-end workflow, we introduce an algorithm that estimates that human's 3D geometry over time from a set of 2D images and develop a 3D-aware image-based rendering approach that embeds the sculpture back into the scene. By automating the process, our system takes motion sculpture creation out of the realm of professional artists, and makes it applicable to a wide range of existing video material. By providing viewers with 3D information, motion sculptures reveal space-time motion information that is difficult to perceive with the naked eye, and allow viewers to interpret how different parts of the object interact over time. We validate the effectiveness of this approach with user studies, finding that our motion sculpture visualizations are significantly more informative about motion than existing stroboscopic and space-time visualization methods.Comment: UIST 2018. Project page: http://mosculp.csail.mit.edu

MyStyle: A Personalized Generative Prior

We introduce MyStyle, a personalized deep generative prior trained with a few shots of an individual. MyStyle allows to reconstruct, enhance and edit images of a specific person, such that the output is faithful to the person's key facial characteristics. Given a small reference set of portrait images of a person (~100), we tune the weights of a pretrained StyleGAN face generator to form a local, low-dimensional, personalized manifold in the latent space. We show that this manifold constitutes a personalized region that spans latent codes associated with diverse portrait images of the individual. Moreover, we demonstrate that we obtain a personalized generative prior, and propose a unified approach to apply it to various ill-posed image enhancement problems, such as inpainting and super-resolution, as well as semantic editing. Using the personalized generative prior we obtain outputs that exhibit high-fidelity to the input images and are also faithful to the key facial characteristics of the individual in the reference set. We demonstrate our method with fair-use images of numerous widely recognizable individuals for whom we have the prior knowledge for a qualitative evaluation of the expected outcome. We evaluate our approach against few-shots baselines and show that our personalized prior, quantitatively and qualitatively, outperforms state-of-the-art alternatives.Comment: Project webpage: https://mystyle-personalized-prior.github.io/, Video: https://youtu.be/QvOdQR3tlO

Bias Correction for Retrieval of Atmospheric Parameters from the Microwave Humidity and Temperature Sounder Onboard the Fengyun-3C Satellite

The microwave humidity and temperature sounder (MWHTS) on the Fengyun (FY)-3C satellite measures the outgoing radiance from the Earth’s surface and atmospheric constituents. MWHTS, which makes measurements in the isolated oxygen absorption line near 118 GHz and the vicinity of the strong water vapor absorption line around 183 GHz, can provide fine vertical distribution structures of both atmospheric humidity and temperature. However, in order to obtain the accurate soundings of humidity and temperature by physical retrieval methods, the bias between the observed and simulated radiance calculated by the radiative transfer model from the background or first guess profiles must be corrected. In this study, two bias correction methods are developed through the correlation analysis between MWHTS measurements and air mass identified by the first guess profiles of the physical inversion; one is the linear regression correction (LRC), and the other is the neural network correction (NNC), representing the linear and nonlinear relationships between MWHTS measurements and air mass, respectively. The correction methods have been applied to MWHTS observed brightness temperatures over the geographic area (180° W–180° E, 60° S–60° N). The corrected results are evaluated by the probability density function of the differences between corrected observations and simulated values and the root mean square errors (RMSE) with respect to simulated observations. The numerical results show that the NNC method has better performance, especially in MWHTS Channels 1 and 7–9, whose peak weight function heights are close to the surface. In order to assess the effects of bias correction methods proposed in this study on the retrieval accuracy, a one-dimensional variational system was built and applied to the MWHTS brightness temperatures to estimate the atmospheric temperature and humidity profiles. The retrieval results also show that NNC has better performance. An indication of the stability and robustness of the NNC method is given, which suggests that the NNC method has promising application perspectives in the physical retrieval

Fusion Retrieval of Sea Surface Barometric Pressure from the Microwave Humidity and Temperature Sounder and Microwave Temperature Sounder-II Onboard the Fengyun-3 Satellite

Both the Microwave Humidity and Temperature Sounder (MWHTS) and the Microwave Temperature Sounder-II (MWTS-II) operate on the Fengyun-3 (FY-3) satellite platform, which provides an opportunity to retrieve the sea surface barometric pressure (SSP) with high accuracy by fusing the observations from the 60 GHz, 118.75 GHz, and 183.31 GHz channels. The theory of retrieving SSP using passive microwave observations is analyzed, and the sensitivity test experiments of MWHTS and MWTS-II to SSP as well as the test experiments of the contributions of MWHTS and MWTS-II to SSP retrieval are carried out. The theoretical channel combination is established based on the theoretical analysis, and the SSP retrieval experiment is carried out based on the Deep Neural Network (DNN) for the theoretical channel combination. The experimental results show that the retrieval accuracy of SSP using the theoretical channel combination is higher than that of MWHTS or MWTS-II. In addition, based on the test results of the contributions of MWHTS and MWTS-II to the retrieval SSP, the optimal theoretical channel combination can be built, and can further improve the retrieval accuracy of SSP from the theoretical channel combination

Fusion Retrieval of Sea Surface Barometric Pressure from the Microwave Humidity and Temperature Sounder and Microwave Temperature Sounder-II Onboard the Fengyun-3 Satellite

Both the Microwave Humidity and Temperature Sounder (MWHTS) and the Microwave Temperature Sounder-II (MWTS-II) operate on the Fengyun-3 (FY-3) satellite platform, which provides an opportunity to retrieve the sea surface barometric pressure (SSP) with high accuracy by fusing the observations from the 60 GHz, 118.75 GHz, and 183.31 GHz channels. The theory of retrieving SSP using passive microwave observations is analyzed, and the sensitivity test experiments of MWHTS and MWTS-II to SSP as well as the test experiments of the contributions of MWHTS and MWTS-II to SSP retrieval are carried out. The theoretical channel combination is established based on the theoretical analysis, and the SSP retrieval experiment is carried out based on the Deep Neural Network (DNN) for the theoretical channel combination. The experimental results show that the retrieval accuracy of SSP using the theoretical channel combination is higher than that of MWHTS or MWTS-II. In addition, based on the test results of the contributions of MWHTS and MWTS-II to the retrieval SSP, the optimal theoretical channel combination can be built, and can further improve the retrieval accuracy of SSP from the theoretical channel combination

Neural Network-Based Distributed Finite-Time Tracking Control of Uncertain Multi-Agent Systems With Full State Constraints

This paper addresses the distributed tracking control problem of pure-feedback multi-agent systems with full state constraints under a directed graph in finite time. By introducing the nonlinear mapping technique, the system with full state constraints is converted into the form without state constraints. Furthermore, by combining fractional dynamic surface and radial basis function neural networks, a novel finite-time adaptive tracking controller is conducted recursively. In light of Lyapunov stability theory, it is proven that all signals of multi-agent systems are semi-globally uniformly ultimately bounded in finite time and the full states satisfy the constraints. Lastly, numerical simulations are supplied to demonstrate the effectiveness of the proposed control strategy